PsL Monthly 1993 December/PSL Monthly Shareware CD-ROM (December 1993).iso/prgmming/dos/pascal/heap.com/HEAP.DOC

Text File | 1989-03-11 | 16.2 KB | 370 lines

              Advanced Heap Management for Turbo Pascal 5.0
                            Version 5.0
                           March 8, 1989

Overview
--------------------------------------------------------------------------
Turbo Pascal's heap is one of the most useful and powerful features of the
language. By using the heap, programs can access all 640K of DOS memory in a
completely dynamic fashion. With power comes responsibility, however. Managing
pointers to the heap is one of the trickiest subjects in Pascal. And as time
goes by, the 640K of DOS memory doesn't seem all that big -- accessing even
more memory is desirable.

This collection of Turbo Pascal units and utilities offers methods for
managing and extending the heap. It includes the following:

  o a patch to TPC.EXE so that the compiler will generate an interrupt to a
    user-supplied routine after dereferencing each pointer.
  o a unit that checks for dereferencing an invalid pointer.
  o a unit that transfers control of the New, Getmem, Dispose, and FreeMem
    procedures to user-supplied routines.
  o a unit that logs various information about the heap to disk at
    convenient points in a program.

Even these are just scratching the surface of what's possible. Further
possibilities include: a completely transparent heap manager that uses either
normal RAM, disk or expanded memory for its raw material; and a handle-based
pointer system that allows heap compaction. We may develop these ourselves as
time becomes available, but we'd like to hear about anything that you develop
along these lines in the meantime.

This collection of utilities is an updated version of those previously
written for Turbo Pascal 4.0.


Patching TPC.EXE
--------------------------------------------------------------------------
The patching program HEAPPAT5 makes a small but important change to TPC.EXE
(version 5.0 only). Once the change is in place, the compiler is capable of
generating an interrupt each time a pointer is dereferenced. When the
interrupt occurs, a user-supplied routine takes control, thus allowing a
program to validate or modify the pointer value.

To apply the patch, compile the supplied program HEAPPAT5.PAS, and run it while
a copy of TPC.EXE is in the current directory. HEAPPAT5 modifies about 75
bytes of the command line compiler (most of which are required to relocate a
particular table so that it will hold two more bytes). Note that the patcher
works only on TPC, not on TURBO.EXE. While it would be possible to make the
same patch to TURBO.EXE, we rarely use the integrated compiler much ourselves,
and it is linked differently enough to make finding the patch points a
time-consuming chore.

HEAPPAT5 assures that the compiler version you have is Turbo Pascal 5.00. If
the file size is wrong, or if any of the locations to be patched contain
unexpected data, HEAPPAT5 will halt with an error message. If this happens, be
sure to restore a clean copy of TPC.EXE from a backup, since a partial patch
may have occurred.

After the patch is successfully completed, the compiler's behavior is changed
in the following manner:

  0. The compiler version will be reported as "5.0P" whenever TPC writes its
  copyright message.

  1. A new compiler switch directive will exist: {$P+} or {$P-}. The default
  state of this switch is {$P-}. When the switch is turned ON, the compiler
  will generate an interrupt $66 after each pointer dereference. The switch
  may be turned on and off as desired in the source code, at the DOS command
  line, or in a TPC.CFG file. Like other Turbo compiler directives, the switch
  returns to its default state at the beginning of each unit.

  It is your program's responsibility to install an interrupt handler prior to
  the first occurrence of int $66 in the program. The supplied unit BADPTR
  automatically installs such a handler for you.

  Interrupt $66 is one of the user-definable interrupts described by IBM. If
  this interrupt conflicts with your application, you may change HEAPPAT5.PAS
  to use a different interrupt. To do so, simply change the constant
  DerefInterrupt in HEAPPAT5.PAS, recompile the program and patch a fresh copy
  of the compiler. User-definable interrupts range from $60 to $66. ($67 is
  used for EMS, so it should be avoided here.)

  If you change the interrupt, be sure to modify any related interrupt handler
  (such as the one in BADPTR) as well.

  2. The patch has one negative side effect. The patched compiler will no
  accept the /$M (stack and heap settings) directive at the command line or in
  the TPC.CFG file. The {$M} source directive is available as usual; the
  memory settings should generally be written in the source code anyway.

  3. Upon entry to your interrupt handler, ES:DI will hold the current value
  of the pointer being dereferenced. The interrupt handler must preserve the
  values of AX, BX, CX, DX, SI, DS, SS, BP, and SP, and it must return either
  the same value of ES:DI or a value appropriately mapped by a virtual memory
  manager.

While a Turbo Pascal Interrupt procedure will work correctly for the pointer
handler, the performance penalty may be unacceptable. For best performance,
the int $66 handler should be written primarily in assembly language. See
BADPTR.ASM for a simple example of such a handler.

The following fragment shows the code generated by the patched compiler when a
pointer P is a global variable, in this case a pointer to an integer.

                 P^ := 1;
        C43E0000       LES    DI,[P]                ;load pointer into ES:DI
        CD66           INT    66                    ;call user routine
        26C7050100     MOV    WORD PTR ES:[DI],0001 ;assign value

An easy rule to remember is this: for each appearance of a dereferencing caret
(^) in your source code, an interrupt will be generated. Of course, this
occurs only when the $P+ directive is in effect for that statement.


Using BADPTR
--------------------------------------------------------------------------
BADPTR is a unit that works in conjunction with the HEAPPAT5-patched compiler.
If you compile with the patched compiler, you must be sure that you USE the
unit BADPTR (or another unit with an int $66 handler), or your program may
crash unexpectedly. (Interrupt $66 often points to an IRET by default, but
that is not always the case.) You should USE BADPTR early in the USES
statement of the main program. If you USE BADPTR and then don't compile with
the patched compiler, no harm will result.

BADPTR automatically installs an interrupt handler that is invoked whenever a
pointer is dereferenced and the $P+ directive is active. This interrupt
handler checks that the pointer refers to the normal Turbo Pascal heap,
between HeapOrg and the top of the free list. If the pointer falls into this
range, the interrupt handler returns and the program proceeds. If the pointer
is outside of the normal heap, BADPTR calls an error routine, which writes the
value of the pointer as well as the relative code address where the error
occurred, and then halts. Note that NIL pointers will always fail BADPTR's
test. You can use Turbo's find runtime error facility to correlate the error
address to a source position.

If you are using pointers not allocated by New or GetMem, there are two ways
to keep BADPTR from reporting a false error. A common example of such a
pointer would be one pointing to the DOS command line, initialized with P :=
Ptr(PrefixSeg, $80). First, you could assure that the $P+ directive is not
active wherever you dereference such a pointer. Second, BADPTR interfaces two
WORD variables that allow you to determine the acceptable range for pointers.
HeapBot is the lowest acceptable pointer segment, and HeapTop is the highest.
BADPTR initializes these to the range of the normal Turbo heap. Suppose you
wanted to accept any pointer but the NIL pointer. In that case you could
assign:

  HeapBot := $0001;
  HeapTop := $FFFF;

BADPTR uses simple WriteLn statements to report an error. If this technique is
not appropriate for a particular application, modify the procedure BadPointer
in BADPTR.PAS. Upon entry to BadPointer, the system variable ErrorAddr
contains the PSP-relative address of the code causing the error, and BadP
contains the faulty pointer. Generally, BadPointer should halt without
returning. If it does not halt, execution will continue after the interrupt,
using the pointer originally supplied in ES:DI. The BadPointer procedure
cannot change the actual pointer value.

If you change HEAPPAT5 to use a different interrupt, be sure to make the
corresponding change in BADPTR as well.


Using GRABHEAP
--------------------------------------------------------------------------
GRABHEAP is a unit that allows a program to take control of memory allocation
and deallocation functions normally handled by the system unit: NEW, GETMEM,
DISPOSE, and FREEMEM. To offer complete control, GRABHEAP interfaces two
procedures:

  procedure CustomHeapControl(GetPtr, FreePtr : Pointer);
    {-Give control of GetMem, New, FreeMem, Dispose to specified procedures}

  procedure SystemHeapControl;
    {-Restore control to the system heap routines}

A program can call CustomHeapControl to transfer control of the system
routines to specified procedures. To do so, pass the addresses of two FAR,
global procedures that match the following declarations:

  {$F+}
  procedure CustomGetMem(var P : Pointer; Size : Word);
  begin
    ...
  end;
  procedure CustomFreeMem(var P : Pointer; Size : Word);
  begin
    ...
  end;
  {$F-}

To set up for this example, an appropriate call would be

  CustomHeapControl(@CustomGetMem, @CustomFreeMem);

Thereafter, any calls that would normally go to New or GetMem are transferred
to CustomGetMem, and calls for Dispose or FreeMem are sent to CustomFreeMem.

The custom heap management routines can perform any needed actions, including
calls to the original system GetMem and FreeMem routines. To call the original
routines, the program must temporarily restore control to the system runtime
library by calling GRABHEAP's SystemHeapControl routine.

Here is an example of CustomGetMem and CustomFreeMem routines that do nothing
but keep a balance sheet of the memory allocated and deallocated in a program:

  var
    TotalAlloc : LongInt;
    HeapMax : Pointer;

  {$F+}
  procedure MyFree(var P : Pointer; Size : Word); forward;

  procedure MyGet(var P : Pointer; Size : Word);
  begin
    Inc(TotalAlloc, Size);              {Update balance sheet}
    SystemHeapControl;                  {Give back heap control temporarily}
    GetMem(P, Size);                    {Use the system routine to allocate}
    if LongInt(HeapPtr) > LongInt(HeapMax) then
      HeapMax := HeapPtr;               {Keep track of heap high water mark}
    CustomHeapControl(@MyGet, @MyFree); {Take over heap control again}
  end;

  procedure MyFree(var P : Pointer; Size : Word);
  begin
    Dec(TotalAlloc, Size);
    SystemHeapControl;
    FreeMem(P, Size);
    CustomHeapControl(@MyGet, @MyFree);
  end;
  {$F-}

  begin
    TotalAlloc := 0;                           {No memory allocated to start}
    HeapMax := HeapOrg;                        {High water mark at base of heap}
    CustomHeapControl(@MyGet, @MyFree);        {Take over heap control}

    ....                                       {Normal program actions}

    WriteLn('Maximum heap usage: ',
            16*(LongInt(seg(HeapMax^))-seg(HeapOrg^)), ' bytes');
    if TotalAlloc <> 0 then
      WriteLn('Allocated memory not freed: ', TotalAlloc, ' bytes');
  end.

GRABHEAP can be put to more powerful uses, of course. For example, based on an
installation flag, it could select among data storage in normal, EMS, or disk
memory. The pointer returned by the custom GetMem routine need not be a normal
pointer, but can instead be an EMS page and offset, or a disk page and offset,
combined into a four byte record. Then, based on the same installation flag, a
deref-interrupt handler can access the data on the appropriate media and
return a pointer to the actual data buffered in memory.


Using HEAPLOG
-------------------------------------------------------------------------
HEAPLOG is a unit that builds on top of the GRABHEAP facility to provide
diagnostic information for programs that make extensive use of the heap. It
keeps a log of all heap allocation and deallocation, and allows the program to
dump this log to disk at any time. By studying the log, one can detect
excessive heap fragmentation and memory that hasn't been deallocated.

You'll get a basic log just by using HEAPLOG in your program. HEAPLOG creates
a file named HEAP.LOG. When a program first starts, a report labeled "Initial"
is written to HEAP.LOG. When the program ends, a report labeled "Final" is
written to HEAP.LOG. The reports themselves will be described momentarily.

HEAPLOG interfaces two procedures that control the logging process:

procedure DumpHeapLog(Msg : string);
  {-Write the current heap log to a file}

procedure ClearLog;
  {-Clear all entries from the log}

DumpHeapLog adds another report to HEAP.LOG, giving it the label passed in
Msg. ClearLog clears HEAPLOG's internal data structures so that succeeding
reports will be relative to that point instead of relative to the beginning of
the program.

The following is an example of a HEAPLOG report.

Initial

MemAvail: 385568
MaxAvail: 385568
HeapPtr : 41DE:0000
HeapCnt : 0
FreeCnt : 0
Filled  : FALSE

Intermediate

MemAvail: 382017
MaxAvail: 380742
HeapPtr : 4309:0002
HeapCnt : 15
FreeCnt : 5
Filled  : FALSE

  Pointer   Size  Allocated at
42CF:0008    115  0000:0AE3
42E9:000F    499  0000:0AE3
41F8:000F    387  0000:07F1
4229:0004    396  0000:07F1
4259:0007    125  0000:07F1
4261:0004    415  0000:07F1
427B:0003    116  0000:07F1
4282:0007    269  0000:07F1
4293:0004    223  0000:07F1
42A4:0009    158  0000:07F1
42AE:0007     42  0000:07F1
42B1:0001    112  0000:07F1
42B8:0001     74  0000:07F1
42BC:000B    301  0000:07F1
42D8:0008    279  0000:07F1

Free start  Size
42D6:000B     29
42A1:0003     54
4242:0000    375
4211:0002    386
41DE:0000    431

Final

MemAvail: 385568
MaxAvail: 385568
HeapPtr : 41DE:0000
HeapCnt : 0
FreeCnt : 0
Filled  : FALSE

HEAPLOG always generates the Initial and Final reports. For programs that
deallocate all their dynamic memory, the Final report should be the same as
the Initial. In the example, the Intermediate report is one generated by
calling DumpHeapLog directly.

Each report shows the values of MemAvail and MaxAvail in bytes, and the
current heap high water mark, HeapPtr. Each also shows the number of separate
blocks allocated on the heap (HeapCnt) and the number of blocks deallocated
and currently on the free list (FreeCnt). If HeapCnt is non-zero, HEAPLOG
shows the value of each allocated pointer, the size of the region it points
to, and the code address where the pointer was allocated. If FreeCnt is
non-zero, HEAPLOG shows the starting address of each free block and its size.

By default, HEAPLOG can track the allocation of up to 1000 pointers in one
run. If the program allocates more than this at one time, the Filled field
will report True. HEAPLOG's capacity can be adjusted by modifying the constant
MaxLog in HEAPLOG.PAS. Note that HEAPLOG itself uses 10 bytes of heap space
for each increment in MaxLog. (HEAPLOG doesn't report its own heap usage,
however.) The performance of calls to GetMem, FreeMem, New, and Dispose will
degrade for large numbers of pointers.


Legalities
--------------------------------------------------------------------------
All programs and documentation in this package are copyright (C) TurboPower
Software, 1988, 1989. All rights reserved. TurboPower Software hereby grants
permission for free distribution of this software, and for use of these units
and techniques within commercial and non-commercial applications. The units
and utilities themselves may not be distributed commercially without obtaining
written permission from TurboPower Software.

We would appreciate hearing about enhancements made to these routines. Contact
Kim Kokkonen at Compuserve ID 72457,2131 or write to:

  TurboPower Software
  P.O. Box 66747
  Scotts Valley, CA 95066